Papers with dictionary learning
Unsupervised Opinion Summarization Using Approximate Geodesics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization are limited due to the scarcity of data. |
| Approach: | They propose a system to perform unsupervised extractive opinion summarization using a dictionary-based representation learning model that generates topical representations of texts. |
| Outcome: | The proposed system achieves strong performance on three opinion summarization datasets. |
Unsupervised Extractive Opinion Summarization Using Sparse Coding (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization rely on human annotations, which may not be feasible. |
| Approach: | They propose to perform opinion summarization in an unsupervised manner by using a dictionary learning algorithm that implicitly captures semantic information from the review text. |
| Outcome: | The proposed algorithm performs well on SPACE and AMAZON datasets and performs controllable summarization to generate aspect-specific summaries using only a few samples. |
Constructing Interpretable Features from Compositional Neuron Groups (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for analyzing LLMs rely on dictionary learning with sparse autoencoders (SAEs) however, SAEs struggle in causal evaluations and lack intrinsic interpretability, as their learning is not explicitly tied to the computations of the model. |
| Approach: | They propose to decompose MLP activations with semi-nonnegative matrix factorization (SNMF) such that the learned features are mapped to their activating inputs, making them directly interpretable. |
| Outcome: | Experiments on Llama 3.1, Gemma 2 and GPT-2 show that SNMF derived features outperform SAEs and a strong supervised baseline on causal steering while aligning with human-interpretable concepts. |